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Abstract— The increasing demand of speed and accuracy for software testing in the industry requires 
advanced testing tools. Efficientsoftware testing can only be done by utilizing necessary and appropriate 
testing methods and frameworks. High quality development can be achieved by using effective automation 
testing. This paper compares various software testing tools andwill aid professionals and researchers in 


this field. 
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I. INTRODUCTION 


Software testing is the process of verifying whether a 
product or application delivers the expected outcome. It 
is done to prevent bugs, improve efficiency of the 
product and reduce costs. The main intent of software 
testing is to find faults and errors [1], so as to solve them 
before the end product reaches the customer. Testing can 
be done manually as well as by using automated tools. 
Manual testing and automated testing are the two 
methods of testing. Manual testing is also called as static 
testing. It is carried out by the tester. Automated testing 
is alsocalled as dynamic testing [2]. Manual testing and 
automated testing both are important testing methods 
but, in most cases, automated testing is considered better 
than manual testing. 


The purpose of this paper is to provide an extensive 
study on various automation tools available in the 
market. A number of testing tools are explained in the 
study along with their pros andcons. The paper is divided 
as follows: Section 1 is the introduction, Section 2 
describes automated testing, Section 3 discusses about 
Test Automation Tools Categories, Section 4 provides 
brief descriptions of automation tools along with their 
advantages and disadvantages, Section 5 presents a 
comparative analysis of various automation testing tools, 
Section 6concludes the study. 


This article can be downloaded from here: www.ijaems.com 


I. AUTOMATED TESTING 


Automated testing is the process of using software to 
compare the actual outcomes of the product with the 
expected outcomes. Automated tests are conducted in 
such a way that they do not require human intervention 
[4]. 


Automated testing can be used in many areas instead of 
manual testing but it cannot replace manual testing 
completely. Both testing methods work together to give 
desired results. The benefits of automation testing over 
manual testing are as follows: 


e Higher Testing Efficiency 

e Greater Accuracy and Reliability 

e  Reusability and Repeatability of Test Scripts 
e Improved Test Coverage 

e Simulation of User Environment 

e Boost in Team Morale 

e Higher ROI: Saves Time and Costs 

e Volume and Simultaneity 


e = Early Detection of Bugs [3] 
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Advantages Disadvantages 


It is more efficient and does | Expensive than manual 
testing quickly. testing. 


Increases number of tests as | Choosing the right tools 
various tools can run takes time and workforce. 


simultaneously. 


It supports TestCafe, 
Cypress, Puppeteer, 

Selenium and Playwright | real devices. 
for automated cross 


It supports only emulators 
and simulators and not 


browser testing. 


Bugs can be found quickly. | Proficiency in languages for 


scripts is needed. 


Advantages Disadvantages 


Results are more accurate Knowledge of tools is 


than manual testing. required. 


Wide range of device and | Not able to execute end- 
OS supported. to- end and regression 
testing. 


HMI. TEST AUTOMATION TOOLS 
CATEGORIES 


Software testing automation tools are categorized as 
follows: Unit Testing Tools, Test Management 
Tools, Code Coverage Tools, Functional Testing 
Tools and Performance Testing Tools. 


7 Test 
Unit Testing Functional Performance 
Tools Testing Tools Daen Testing Tools 
Code Coverage 
Tools 


Figl : Categories of Test Automation Tools 


1. Test Automation Tools 


An automation tool is a software itself with the help of 


which the actual software in focus can be tested, in other 
words, the automation tool helps and serves as a means 
in doing software testing [5]. Following is the 
description of some common testing automation tools 
available for use: 


1. LambdaTest 


LambdaTest is an automation testing tool used for 
desktop and web applications. It can run selenium 
test automation on desktop, android and iOS mobile 
browsers in languages like Python, Java, JavaScript, 
etc. It allows integrated developer tools to help 
debug issues in live testing, local hosted web testing 
and geolocation testing. 


Does not need additional 
dependencies. 


Cannot produce real world 
conditions to test against. 


5. Avo Assure 


Avo Assure is a 100% no-code and intelligent 
test automation tool. It enables technical as well 
as business users to test across multiple platforms 
like mobile, web, desktop, mainframes etc. 


Advantages Disadvantages 


It is a scriptless Updating an existing mind 


automation tool. map is difficult. 


Easy to learn and user 
friendly. Need to learn 


Report size is big which 
makes it difficult to debug. 
programming languages is 
eliminated. 


2. Katalon Studio 


Katalon Studio is used for automation testing for 
API, desktop, mobile and web. It is an end-to-end 
automation platform built for continuous testing and 
bringing solutions for every industry. It is quick to 
adopt, powerful to scale and has seamless Agile and 
CI/CD Integration. 


Advantages Disadvantages 


No licensing and 
maintenance fees 


Poor community support. 


required. 


Built on Selenium 
framework but does not 


Lack of support of 
programming languages. 


need advanced Supports only 


programming skills. Java/Groovy. 


Advantages Disadvantages 


Variety of frameworks to | There is no native app 
choose from. testing. 


This article can be downloaded from here: www.ijaems.com 


3. Subject7 


Subject7 is a cloud-based and truly codeless test 
automation tool. It is easy to use integrates easily 
with DevOps/ Agile tooling using native plugins, 
open API’s and in-app integrations. 
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Advantages Disadvantages 
Features like cloud test UL is not 
execution, multi browser that user 
testing, database, connection, friendly. 


REST callsare available. 


Does not require extensive 
knowledge of programming 
languages. 


Less online 
resources 
available. 


8. TestComplete 


TestComplete is a GUI automation testing tool 
used to testdesktops, web applications and mobile 
applications. Made for technical as well as non- 
technical users. It uses modern languages like 
Python, JavaScript and VBScript. 


Advantages Disadvantages 


6. IBM Rational Functional Tester 


The main purpose of 


this tool is regression 


testing and automated functional testing. It can 
perform GUI and data-driven testing. IBM RFT 
supports a variety of terminal emulator- based 
and web-based applications. 


Additional cost for extra 
modules and add-ons. 


Allows to choose a variety 
of scripting languages. 


It requires only basic 
programming skills. 


Licensing and 
maintenance fees are 
needed. 


Advantages Disadvantages 

Quick response and good | Very less support of 
support. IBM releases | browsers other than 
frequent updates to fix | Internet Explorer. 

issues. 


There is no set up fees. 


Uses a lot of memory. 


7. Test Studio 


Test Studio is a test automation solution suited 
for GUI, performance, load and API testing. It 


uses languages like C# 


and VB.NET, a central 


object repository andcontinuous integration with 


source control. 


Advantages 


Disadvantages 


It comes with extensive 


support for data-driven 


testing. 


It is not a free tool and is a 
bit costly. 


Without writing a single 
line of code, automated 
tests can be run on devices 
and emulators. 


It needs powerful 
computers to run all its 


capabilities. 


Tools 


LambdaTest 
Test Complete 


KatalonStudio 
Subject7 


Kobiton 
AVO Assure 


IBM RFT 
Test Studio 


Price (Annually) 
Free/Paid plansavailable 
$3000/$5000 


Free/Paid plansavailable 
Given exclusivelyby software provider 


$4000/$9000 


Given exclusivelyby software provider 
Given exclusivelyby software provider 
$2500/$3500 


This article can be downloaded from here: www.ijaems.com 
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4. Kobiton 


Kobiton’s scriptless test automation makes testing 
easy by creating automated tests from manual tests. It 
creates scripts which execute on over 350 devices. It 
supports XCUI, Selenium, Appium etc. for 


scripting. 


IV. COMPARATIVE ANALYSIS OF VARIOUS 
AUTOMATION TESTING TOOLS 


There a lot of Automation Tools available in the market. 
The selection of the best tool for one’s necessity is the 
difficult task. To reduce the labour of searching for the 
best tool, below is a detailed comparison of the various 
automation solutions. 


4.1 On basis of price and platform 


4.2 On basis of languages, coding skills and tested 
apps Languages are sometimes necessary in automated 
testing assome software work using coding. Necessary 
skills arerequired by user for the proper working of 
these tools. The skillset may range from minimum 
skills to good/advanced level. The following is the 
comparison according to theseparameters: 


Platform 
Windows/Linux/Mac 
Windows 
Windows/Mac 
Windows 
Windows/ Linux /Mac 
Windows/Mac 


Windows/Linux/Mac 
Windows 
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Tools SupportedLanguages 
LambdaTes Java/ Python/ Ruby/ PHP/ C# 
t 
TestComple VB/ JavaScript/C++/ C#/ 
le Delphi/ 
Angular/ Rubyon Rails/ PHP 
Katalon 
Studio Java/ Groovy 
Subject7 Codeless automation 
Kobiton C#/ Java/ Ruby/NodeJS/ PHP/ Python 
AVO Codeless automation 
Assure 
HTML/ Java/Windows/ 
.NET/ Visual Basic/ Silverlight/ Eclipse/ 
eae eel SAP/Siebel/ Flex/ Ajax/ Dojo/ GEF/ 
PowerBuilder 
HTML/ AJAX/ 
Test Studio Silverlight/ ASP.NET MVC/ 


JavaScript/ WPF/ Angular/React/ Blazor 


vV. CONCLUSION 


There are many automation tools in the market. The 
main purpose of these tools is to reduce time and 
workload for completing a task. Unfortunately, the 
features discussed in this study do not alone meet the 
requirements of the industry. Each tool has its own 
unique features and can be used effectively if chosen 
wisely for particular tasks. This study provides detailed 
explanation of various automation tools so as to ease the 
selection of automation tool and framework by 
professionals 
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CodingSkills TestedApps 
Good skills 
needed ce 
Minimumskills Wepy Mobile 
Desktop 
Minimumskills Web/ Mobile 
Minimum/No skills Web 
Good skills 
needed Wee 
E : Web/ Android/ 
Minimum/No skills iPhone/ iPad 
Good skills 
needed Web/ Desktop 
Good skills 
needed Web 


Includes Complete Guidelines, Checklists, and Templates, 
Third Edit. 2007. 
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